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GAO 


United  States 

General  Accounting  Office 

Washington,  D.C.  20548 


Accounting  and  Information 
Management  Division 

B-261668 

August  22, 1995 

The  Honorable  John  Glenn 
Ranking  Minority  Member 
Committee  on  Governmental  Affairs 
United  States  Senate 

Dear  Senator  Glenn: 

As  you  know,  one  of  the  Environmental  Protection  Agency’s  (epa) 
responsibilities  is  to  administer  a  national  program  for  the  safe 
management  of  hazardous  waste.  Pursuant  to  the  Resource  Conservation 
and  Recovery  Act  (rcra),  epa  has  granted  authority  to  most  states  to 
implement  key  requirements  of  the  act,  including  permitting,  inspecting, 
monitoring,  and  enforcement. 

epa  developed,  and  in  1991  began  operating,  the  Resource  Conservation 
and  Recovery  Information  System  (rcris)  with  the  goal  of  helping  both  the 
federal  government  and  the  states  better  manage  the  hazardous  waste 
program.  Intended  to  overcome  serious  concerns  about  the  system  it 
replaced,  rcris’  primary  objectives  were  to  (1)  provide  individual 
databases  for  each  state  to  use  in  managing  the  program,  (2)  be  user 
friendly,  (3)  be  flexible  enough  to  meet  federal  requirements,  as  well  as  the 
specific  needs  of  individual  states  and  epa  regions,  and  (4)  provide  a 
mechanism  for  maintaining  highly  reliable  data. 

In  your  October  13,  1994,  letter  to  us,  you  expressed  concern  about 
whether  rcris  is  meeting  the  needs  of  epa  and  the  states.  As  agreed  with 
your  office,  our  objectives  were  to  determine  whether  the  system  is 
meeting  its  primary  objectives  and,  if  not,  the  effect  any  shortfalls  have 
had  on  rcra  implementation.  Details  on  our  scope  and  methodology  are 
provided  in  appendix  I. 


Results  in  Brief 

[/Otxo^jz^  i 

0-| 


•  _ ^ 


rcris  falls  short  of  meeting  its  overall  goal  of  helping  epa  and  the  states 
manage  the  hazardous  waste  program  because  it  has  not  met  three  of  its 
primary  objectives.  While  rcris  gives  the  states  their  own  individual 
databases,  data  entry  and  retrieval  are  so  difficult  that  most  users  do  not 
rely  on  the  system.  Moreover,  while  the  system  was  intended  to  be  flexible 
enough  to  meet  individual  needs,  most  users  do  not  attempt  to  do  this 
because  of  the  difficulties  using  and  modifying  the  system.  Furthermore, 
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the  data  in  rcris  are  not  reliable,  forcing  users  to  manually  verify  the  data 
before  it  can  be  used. 

rcris’  shortfalls  have  not  significantly  affected  rcra  implementation, 
primarily  because  the  system  is  not  relied  on  as  a  key  tool  for  managing 
the  program.  Instead,  users  rely  on  systems  they  have  independently 
developed  as  well  as  other  work-arounds  that  have  been  created. 
Recognizing  that  federal  and  state  users’  needs  are  not  being  met  through 
rcris,  epa  recently  began  reassessing  information  needs  and  systems 
support  for  the  rcra  program. 


Background 


Congress  enacted  rcra  to  protect  human  health  and  the  environment  and 
to  conserve  energy  and  material  resources.  In  implementing  rcra,  epa  is 
responsible  for  developing  federal  hazardous  waste  management  and 
disposal  requirements  and  for  ensuring  compliance  with  these 
requirements.  Two  offices  within  epa  share  responsibility  for  rcra — the 
Office  of  Solid  Waste  (osw)  and  the  Office  of  Enforcement  and  Compliance 
Assurance. 

epa  may  authorize  states1  to  implement  their  own  rcra  programs,  or 
portions  thereof,  provided  that  their  programs  meet  federal  requirements. 
Currently,  46  states,  the  District  of  Columbia,  and  Guam,  have  been 
authorized  to  implement  at  least  some  portion  of  rcra.  States  that  are 
authorized  receive  annual  grants  from  epa  and  are  subject  to  epa  oversight. 
In  fiscal  year  1994,  epa  gave  approximately  $92  million  in  rcra  grant 
money  to  these  states  and  territories. 

To  help  manage  rcra,  epa  and  authorized  states  collect  and  manage  data 
on  more  than  250,000  hazardous  waste  handlers,  including  210,900  small 
quantity  generators;  15,300  large  quantity  generators;  19,700  transporters; 
and  4,300  treatment,  storage,  and  disposal  facilities.  Day-to-day  rcra 
activities  include  assigning  hazardous  waste  handler  identification 
numbers;  issuing  permits;  tracking  inspections,  violations,  and 
enforcement  actions;  and  maintaining  other  related  data  to  assess  the 
extent  to  which  facilities  are  complying  with  rcra  regulations. 

In  1985,  to  overcome  problems  with  the  information  system  it  was  using, 
known  as  the  Hazardous  Waste  Data  Management  System,  epa  began 
developing  rcris.  Both  the  states  and  epa  had  reported  that  the  Hazardous 
Waste  Data  Management  System  was  not  adequately  supporting  the 


lrThese  include  all  50  states  as  well  as  the  District  of  Columbia  and  five  territories. 
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program  in  many  areas,  including  permitting,  compliance  monitoring,  and 
enforcement.  In  addition,  the  system  was  criticized  as  cumbersome, 
difficult  to  use,  and  confusing,  and  the  database  was  criticized  as 
excessively  complex. 

The  overall  goal  of  rcris  was  to  help  both  the  federal  government  and  the 
states  better  manage  the  hazardous  waste  program.  The  National 
Governors’  Association  assisted  epa  in  identifying  state  needs  for  rcris.  epa 
determined,  based  on  information  collected  by  the  Association,  that  for 
rcris  to  be  successful,  the  system  had  to  meet  the  following  four  key 
objectives:  (1)  provide  individual  databases  for  each  state,  (2)  be  user 
friendly,  (3)  be  flexible  enough  to  meet  federal  requirements,  as  well  as  the 
specific  needs  of  individual  states,  and  (4)  provide  a  mechanism  for 
maintaining  highly  reliable  data. 

rcris  became  operational  nationwide  in  1991.  The  system  comprises  four 
key  modules  that  mirror  the  major  rcra  program  activities.  Table  1 
provides  the  name  and  purpose  of  these  modules. 


Table  1 :  Key  RCRIS  Modules  and 
Intended  Purpose 


Module 

Purpose 

Handler  Identification 

Maintain  basic  data  identifying  and 
describing  handlers  so  they  can  be 
tracked  and  monitored 

Permitting,  Closure,3  and  Post-Closure 

Maintain  detailed  information  about 
hazardous  waste  treatment;  storage  and 
disposal  processes;  and  permitting, 
closure,  and  post-closure  activities 

Compliance  Monitoring  and  Enforcement 

Maintain  information  on  inspections, 
violations,  and  enforcement  actions 

Corrective  Action 

Track  specific  corrective  action  information 
needed  to  regulate  facilities  with 
hazardous  waste  releases 

aClosure  occurs  when  hazardous  wastes  are  no  longer  accepted  and  the  facility  is  prepared  to 
cease  operations. 

Although  osw  did  not  maintain  information  about  rcris  operational  costs,  a 
recent  report  by  the  epa  Inspector  General  stated  that  fiscal  year  1993 
operational  costs  were  $7.5  million.2  This  figure,  however,  excludes  epa’s 
and  states’  personnel  costs  for  time  spent  (1)  entering,  retrieving,  and 
manipulating  data  in  rcris  and  (2)  developing  and  operating  other  systems 


Management  of  Application  Software  Maintenance  at  EPA  (E1NMF3-15-0072-5100240,  Mar.  31, 1995). 
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and  work-arounds  to  compensate  for  rcris  inadequacies.  Officials  in  osw 
said  they  did  not  know  these  costs. 

Scope  and 
Methodology 

Our  work  was  performed  at  several  offices  at  epa  headquarters,  including 
the  Office  of  Solid  Waste,  the  Office  of  Enforcement  and  Compliance 
Assurance,  and  Office  of  Inspector  General.  These  offices  were  located  in 
Washington,  D.C.,  and  Arlington,  Virginia.  We  also  worked  at  epa  regional 
offices  in  Atlanta,  Georgia;  Dallas,  Texas;  and  San  Francisco,  California; 
and  at  state  offices  responsible  for  rcra  in  Tallahassee,  Florida;  Jackson, 
Mississippi;  Little  Rock,  Arkansas;  Austin,  Texas;  Carson  City,  Nevada;  and 
Sacramento,  California. 

We  conducted  our  review  from  August  1994  through  May  1995,  in 
accordance  with  generally  accepted  government  auditing  standards.  We 
requested  comments  on  a  draft  of  this  report  from  the  Administrator  of 
epa.  The  Director  for  the  Office  of  Solid  Waste  provided  us  with  comments 
on  July  18,  1995.  We  have  incorporated  these  comments  where 
appropriate. 

RCRIS  Is  Not  Meeting 
User  Needs 

rcris  falls  short  of  meeting  its  overall  goal  of  helping  epa  and  the  states 
manage  the  hazardous  waste  program  because  it  has  not  met  three  of  its 
primary  objectives.  While  rcris  provides  individual  databases  for  states  to 
use  to  maintain  their  data,  the  system  is  difficult  to  use,  difficult  to  tailor  to 
meet  individual  state  and  epa  regional  needs,  and  does  not  provide  a 
mechanism  for  maintaining  highly  reliable  data. 

States  Can  Maintain  Their 
Own  RCRIS  Databases 

rcris  provides  two  databases  for  each  state — one  maintained  by  the  state 
and  one  maintained  by  the  epa  regional  office.  These  databases  are 
intended  to  support  day-to-day  rcra  activities  carried  out  by  the  states  and 
epa  regional  offices.  Extracts  from  these  databases  are  merged  to 
construct  a  database  of  required  information  for  regional  oversight.  A 
national  oversight  database  for  epa  headquarters  and  national  reporting  is 
then  formed  by  merging  extracts  from  the  regional  merged  databases. 

RCRIS  Is  Difficult  to  Use 

rcris  is  not  easy  to  use,  particularly  with  regard  to  data  entry  and  retrieval. 
Data  entry  is  cumbersome  largely  because  of  the  design  of  the  four  key 
rcris  modules.  For  example,  each  module  uses  numerous  data  entry 
screens,  many  of  which  allow  the  user  to  modify  data  in  only  a  few  data 
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RCRIS  Does  Not  Satisfy 
Individual  State  and  EPA 
Regional  Needs 


fields  at  one  time.  The  rcris  user  manual  shows  that  a  user  could  possibly 
interact  with  up  to  150  data  entry  screens  to  add,  modify,  or  delete  data. 

Because  of  this  design,  users  are  often  required  to  switch  between 
modules,  screens,  and  files — even  when  making  simple  changes  to  the 
data.  For  example,  to  modify  a  hazardous  waste  handler’s  information  and 
add  data  for  an  inspection  of  a  specific  facility,  a  user  must  access  the 
handler  identification  module,  enter  the  handler’s  identification  number, 
update  the  handler  information,  exit  the  handler  module,  enter  the 
compliance  monitoring  and  enforcement  module,  reenter  the  handler’s 
identification  number,  and  then  enter  the  inspection  information. 

Data  entry  is  also  difficult  because  codes  are  used  extensively  to  represent 
much  of  the  information  that  is  recorded.  These  codes  are  used  to 
describe  various  aspects  of  a  hazardous  waste  handler,  such  as  the 
process  code  that  specifies  the  current  waste  treatment,  storage,  or 
disposal  process  being  used,  rcris’  four  key  modules  include  about  875 
codes.  The  rcris  user  manual  notes  that  understanding  the  majority  of  the 
codes  is  essential  for  successful  data  entry,  maintenance,  processing,  and 
reporting. 

Users  have  also  found  data  retrieval  difficult.  While  users  can  obtain 
several  standard  reports  from  the  system,  these  reports  are  voluminous. 
For  example,  up  to  four  cardboard  boxes  are  needed  to  hold  one  standard 
report  on  facilities.  Consequently,  users  need  to  query  the  system  to  obtain 
specific  information,  but  creating  customized  reports  is  very  difficult.  For 
example,  to  obtain  information  on  a  specific  facility’s  history  or  status 
directly  from  the  system,  a  user  must  first  know  which  files  contain  data 
on  the  facility.  The  user  must  then  query  each  file  to  obtain  the  desired 
data.  Finally,  the  user  must  know  how  to  merge  the  extracted  data  to 
produce  a  single  report.  Users  in  the  six  states  and  the  three  epa  regions 
we  visited  said  that  creating  these  customized  reports  is  very  difficult. 


Another  primary  objective  for  rcris  was  that  it  be  flexible  enough  to  allow 
states  and  epa  regions  to  tailor  the  system  to  meet  their  individual  program 
needs.  However,  users  in  five  of  the  six  state  offices  and  all  three  of  the 
regional  offices  we  visited  stated  that  rcris  has  not  met  their  individual 
needs.  Many  of  these  users  said  they  have  no  desire  to  tailor  rcris  to  meet 
their  needs  because  the  system  is  just  too  difficult  to  use. 
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RCRIS  Data  Are  Not 
Reliable 


We  identified  users  in  two  epa  regions  who  said  they  had  been  able  to 
better  meet  some  individual  needs  using  rcris.  For  example,  because  rcris’ 
standard  reports  do  not  include  the  precise  information  needed,  these 
users  developed  customized  query  programs  and  reports,  and  have  made 
them  available  to  other  users  across  the  country.  These  users  said  they 
were  able  to  do  this  because  they  invested  a  significant  amount  of  time  in 
learning  the  intricacies  of  the  system  and  had  become  expert  in  using  it. 

In  addition,  because  almost  all  the  states  were  using  or  developing  their 
own  systems,  epa  wanted  them  to  be  able  to  extract  the  required  data  from 
their  systems  and  submit  it  to  rcris  electronically.  Although  states  can  do 
this,  state  users  told  us  that  creating  conversion  software  to  reformat  their 
data  into  rcris’  data  structure  and  then  modifying  the  conversion  software 
each  time  epa  changes  rcris’  data  structure  can  be  extremely  difficult. 
Consequently,  only  one  state  transmits  any  data  to  rcris  electronically. 


rcris  does  not  provide  a  mechanism  for  maintaining  highly  reliable  data 
While  epa  does  not  require  data  qualify  reviews  of  the  data  in  rcris,  the  few 
ad  hoc  reviews  that  have  been  conducted  have  uncovered  significant 
problems.  For  example,  a  Region  VI  (Dallas)  review  of  facility 
demographic  data  on  about  45,000  hazardous  waste  handlers  identified 
over  15,000  errors  on  the  location  of  the  handlers.  Similarly,  Region  IV 
(Atlanta)  found  anecdotal  evidence  of  discrepancies  between  the  data  in 
rcris  and  the  information  in  hard  copy  files.  According  to  epa  officials, 
states  have  corrected  most  of  these  discrepancies.  However,  users  in  all 
regional  and  state  offices  we  visited  consistently  expressed  concerns 
about  the  quality  of  rcris  data. 

In  addition,  as  part  of  a  review  of  whether  land  disposal  facilities  were 
complying  with  groundwater  monitoring  requirements,  we  found  rcris 
data  to  be  unreliable.  We  selected  a  random  sample  of  30  facilities  out  of 
rcris’  universe  of  1,427  classified  by  epa  as  land  disposal  facilities.  For 
each  sampled  facility,  we  reviewed  data  in  14  data  fields  that  were  most 
relevant  to  groundwater  monitoring  requirements.  We  estimated  minimum 
error  rates  ranging  from  8  to  37  percent  for  eight  of  the  data  fields.3  These 
data  provided  information  on  the  types  of  violations  detected  and 
enforcement  actions  taken. 

These  data  reliability  problems  can  be  attributed,  in  part,  to  a  lack  of 
controls  over  the  quality  of  data  when  epa  converted  from  the  predecessor 


3See  appendix  II  for  details  on  the  error  rates. 
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system  to  rcris.  Although  epa  recognized  that  the  conversion  would  result 
in  inaccurate  data,  it  did  not  require  verification  and  correction  of  the 
data — a  generally  accepted  practice.  Instead,  epa  loaded  the  inaccurate 
data  and  then  left  it  up  to  each  state  and  region  to  perform  a 
post-conversion  assessment  of  data  quality.  Further,  although  epa 
(1)  provided  software  to  help  the  states  and  regions  detect  data  quality 
errors  after  the  conversion,  (2)  set  aside  5  percent  of  state  grant  funding  to 
support  data  cleanups,  and  (3)  set  a  90-percent  accuracy  goal  for  the 
converted  data,  the  agency  allowed  users  to  choose  which  data  they  would 
check  and  did  not  determine  whether  states  and  regions  had  met  the 
accuracy  goal.  Thus,  officials  in  all  three  regions  we  visited  had  concerns 
about  lingering  data  quality  problems  because  of  unverified  data  from  the 
predecessor  system. 

A  second  factor  contributing  to  data  quality  problems  is  epa  headquarters’ 
failure  to  provide  precise  definitions  for  certain  data  elements  in  the 
system.  Officials  in  all  regions  and  five  of  the  six  states  we  visited  said  that 
rcra  data  definitions  are  interpreted  differently  and  have  led  to  differences 
among  states  in  how  data  in  the  same  fields  are  recorded  in  rcris.  For 
example,  while  epa  requires  users  to  identify  facilities  that  bum  hazardous 
waste,  the  agency  does  not  specify  whether  only  a  facility  that  bums 
hazardous  waste  as  a  regular  part  of  its  operations  should  be  considered  a 
bum  facility,  or  if  all  facilities  that  have  ever  burned  waste,  regardless  of 
frequency,  should  be  included.  Headquarters  officials  agreed  that  this  is  a 
critical  problem  and  acknowledged  that  no  one  has  taken  control  of  data 
standardization  issues. 

Another  factor  contributing  to  unreliable  data  is  that  rcris’  front-end  edit 
checks  do  not  effectively  help  ensure  that  data  are  accurate  when  they  are 
entered.  They  do  not  check  whether  data  that  are  entered  are  consistent 
with  other  related  data  fields.  For  example,  while  violations  typically 
result  from  an  inspection,  violation  data  can  be  entered  into  the  rcris 
violation  file  even  if  the  inspection  data  were  never  entered  because  users 
are  not  prompted  to  enter  the  related  inspection.  Therefore,  incomplete 
information  may  remain  in  the  system. 

Finally,  epa  does  not  have  a  data  quality  assurance  program  that  would 
include  establishing  reliability  standards  and  require  periodic  reviews  of 
rcris  data  to  determine  if  they  meet  these  standards,  epa  headquarters 
officials  said  they  do  not  require  regular  data  quality  reviews  because  they 
believe  that  it  is  the  regions’  and  states’  responsibility  to  ensure  data  in  the 
system  are  reliable,  epa  officials  in  the  three  regions  we  visited  told  us  that 
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they  use  a  rcris  software  feature  to  verify  certain  data,  but  that  overall 
data  quality  reviews  have  only  been  done  on  an  ad  hoc  basis,  and  that 
these  reviews  had  focused  on  limited  data  elements. 


RCRIS  Shortfalls  Have 
Not  Significantly 
Affected  the  RCRA 
Program 


Although  rcris  has  not  met  its  original  objectives,  system  problems  have 
not  significantly  affected  the  rcra  program  because  users  do  not  rely  on  it 
as  a  primary  management  tool.  However,  rcris’  problems  have  placed 
additional  burdens  on  epa  and  the  states  to  develop  work  arounds  and 
additional  systems.  All  of  the  regions  and  states  we  visited  were  using  at 
least  one  PC-based  system  to  supplement  rcris.  In  some  cases,  these 
systems  did  not  address  unique  needs,  but  rather  duplicated  the  same 
functions  as  rcris  in  a  format  that  was  easier  to  use. 


EPA  Officials  Do  Not  Rely  While  EPA  officials  stated  that  rcris  has  improved  some  aspects  of  systems 
on  RCRIS  Data  support,  officials  in  all  three  regions  we  visited  and  headquarters, 

including  the  Director  of  the  Office  of  Solid  Waste,  told  us  that  the  system 
is  not  reliable  enough  to  strategically  manage  the  rcra  program.  Some  data 
in  rcris  are  used  for  program  planning,  state  grant  workload  models,  and 
regulatory  development.  However,  rather  than  relying  on  information 
directly  out  of  the  system,  epa  headquarters  officials  said  they  routinely 
send  national-level  rcris  reports  back  to  regional  offices  and  the  states  for 
subsequent  verification  and  correction  before  they  accept  and  use  the 
data,  epa  program  officials  and  staff  also  told  us  that  rcris  does  not 
provide  all  the  information  they  need.  For  example,  these  officials  pointed 
out  that  universe  information — the  total  number  of  hazardous  waste 
handlers  in  particular  classes — cannot  be  obtained  from  rcris.  As  a  result 
of  these  problems,  the  Director  stated  that  he  believes  it  is  necessary  to 
rethink  rcra’s  information  needs,  particularly  with  regard  to  data  needed 
for  environmental  and  management  indicators. 


States  Use  Their  Individual 
Systems 


All  six  of  the  state  locations  we  visited  use  their  own  systems  to  help  them 
meet  their  information  needs.  State  officials  said  they  enter  data  into  rcris 
primarily  to  satisfy  their  rcra  grant  commitments,  which  require  them  to 
devote  specific  resources  to  supporting  rcris  and  entering  required  data 
into  the  system.  The  officials  also  said  they  maintain  their  own  systems 
because  these  systems  are  easier  to  use  and  they  meet  the  state’s 
individual  needs  much  better  than  rcris.  For  example,  California  maintains 
separate  databases  for  managing  its  regulated  universe  of  hazardous  waste 


Page  8 


GAO/AIMD-95-167  EPA’s  Hazardous  Waste  Information  System 


B-261668 

handlers  because  its  data  needs  are  more  detailed  and  broader  in  scope 
than  epa’s. 

EPA  Efforts  to  Address 
Shortfalls 

During  our  review,  epa  acknowledged  rcris’  many  problems.  Further,  osw 
admits  that  it  does  not  know  who  the  system’s  users  are  or  how  frequently 
the  system  is  used.  In  addition,  as  previously  noted,  epa  does  not  know 
how  much  money  is  being  spent  on  rcris  field  support  or  on  developing 
and  maintaining  supplemental  systems. 

Recognizing  rcris’  shortcomings,  osw  has  begun  a  new  strategic 
effort — the  Waste  Information  Needs  (win)  initiative — to  determine  the 
information  and  technology  needs  of  epa  headquarters,  epa  regions,  and 
states,  epa  intends  to  produce  a  plan  addressing  these  needs  by 

September  1995.  In  the  interim,  epa  plans  to  continue  operating  rcris  even 
though  (1)  most  states  we  visited  do  not  use  it  to  meet  their  needs  and 
(2)  epa  acknowledged  that  rcris  data  are  unreliable. 

Conclusions 

rcris  is  providing  few  of  the  benefits  that  were  originally  intended.  The 
system  is  difficult  to  use  and  its  data  are  unreliable.  Consequently,  many  of 
the  users  rcris  was  created  to  serve  do  not  rely  on  the  system  or  the 
information  in  it.  Instead,  states  operate  their  own  systems,  and  epa  uses 
other  means  to  get  reliable  data.  Until  epa  identifies  what  information  it 
needs  to  manage  rcra,  epa  will  continue  to  operate  a  national  system  with 
unknown  costs,  few  benefits,  and  documented  burdens. 

Recommendations 

We  recommend,  as  a  part  of  the  recently  initiated  reassessment  of  rcra 
information  needs,  that  the  Administrator  of  the  Environmental  Protection 
Agency  require  the  Assistant  Administrator  of  the  Office  of  Solid  Waste 
and  Emergency  Response  to 

•  determine  what  information  epa  needs  to  oversee  states’  implementation 
of  RCRA, 

•  develop  clear  data  definitions  for  all  epa  required  data, 

•  develop  a  data  quality  assurance  program  that  establishes  data  reliability 
standards  and  methods  to  ensure  data  reliability,  and 

•  develop  and  implement  a  cost-effective  solution  for  meeting  identified 
needs. 
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In  the  interim,  given  the  burden  rcris  imposes  on  the  states  and  the  lack  of 
epa’s  reliance  on  the  system,  we  recommend  that  the  Administrator  of  the 
Environmental  Protection  Agency  direct  the  Assistant  Administrator  of  the 
Office  of  Solid  Waste  and  Emergency  Response  to  assess  rcris  data 
reporting  requirements  and  eliminate  those  that  are  identified  as 
nonessential. 


Agency  Comments 
and  Our  Evaluation 


In  commenting  on  a  draft  of  this  report,  the  Director  for  the  Office  of  Solid 
Waste  stated  that  epa  generally  agreed  with  our  analyses,  findings,  and 
recommendations.  However,  the  Director  requested  clarification  on  two  of 
our  recommendations. 

First,  our  draft  report  proposed,  as  part  of  the  win  initiative,  that  epa  define 
the  information  needs  of  the  states  that  require  system  support  from  epa  to 
manage  their  programs.  The  Director  stated  that  epa  did  not  know  if  we 
were  recommending  that  the  agency  only  define  information  needs  or  if 
we  were  recommending  that  it  also  supply  system  support  (development 
and/or  operational)  to  those  states.  The  Director  said  epa  realizes  that  it 
cannot  build  one  automated  system  to  support  all  users  and  that  epa  wants 
to  get  out  of  the  business  of  supplying  system  support  to  the  states. 

We  made  this  proposal  because  of  our  concern  that  in  conducting  the  win 
initiative,  epa  would  again  gather  information  requirements  from  state  and 
epa  users,  and  then  develop  a  system  that  it  hoped  would  satisfy  all  of 
these  requirements.  Our  intent  was  for  epa  to  explicitly  identify  which 
states  require  systems  support  in  order  to  narrow  the  requirements  for  the 
new  system  to  only  those  users  who  will  rely  on  the  system.  We  have 
deleted  our  recommendation  based  on  the  Director’s  recognition  that  epa 
cannot  build  one  system  to  meet  federal  and  state  needs;  however,  we 
caution  epa  as  it  proceeds  with  the  win  initiative  to  determine  and  make 
clear  to  the  states  specifically  what  level  of  system  support  will  be 
provided. 

Secondly,  we  proposed  that  epa  consider  eliminating,  while  it  completed 
the  win  initiative,  the  requirement  for  states  to  supply  data  to  rcris.  The 
Director  stated  that  epa  realizes  that  the  burdens  associated  with 
maintaining  rcris  need  to  be  reduced,  but  noted  that  epa  relies  on  such 
data  for  program  implementation  and  management,  epa  did  agree  to 
consider  eliminating,  or  significantly  reducing,  the  requirement  for  states 
to  supply  data  to  rcris. 
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We  did  not  intend  for  epa  to  take  an  all-or-nothing  approach  to  rcris.  In 
epa’s  consideration  of  the  option,  we  expected  that  certain  data  elements 
would  be  identified  as  essential  while  others  could  be  eliminated.  We  have 
modified  our  recommendation  to  reflect  this. 


As  arranged  with  your  office,  unless  you  publicly  announce  the  contents  of 
this  report  earlier,  we  plan  no  further  distribution  until  30  days  from  the 
date  of  this  letter.  At  that  time,  we  will  send  copies  to  the  Administrator  of 
the  Environmental  Protection  Agency,  Director  of  the  Office  of 
Management  and  Budget,  and  interested  congressional  committees. 

Copies  will  also  be  made  available  to  others  upon  request. 

Please  call  me  at  (202)  512-6253  if  you  or  your  staff  have  any  questions 
concerning  this  report.  Other  major  contributors  are  listed  in  appendix  m. 

Sincerely  yours, 


Joel  C.  Willemssen 
Director,  Information  Resources 
Management/Resources,  Community, 
and  Economic  Development 
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Scope  and  Methodology 


To  address  our  objectives,  we  used  a  structured  interview  document  to 
discuss  use  of  the  system  and  its  information  with  regional  and  state  users. 
We  interviewed  officials  and  staff  at  epa  regional  offices  in  Dallas,  Texas; 
Atlanta,  Georgia;  and  San  Francisco,  California.  We  chose  these  locations 
because  they  are  three  of  the  largest  rcra  regions  responsible  for  17  states 
and  3  territories,  and  because  headquarters  officials  believed  they  would 
adequately  represent  the  diversity  in  use  of  rceis. 

We  also  interviewed  representatives  of  six  state  environmental  protection 
offices — the  largest  in  each  of  the  regions — Texas,  Florida,  and 
California — and  the  smallest — Arkansas,  Mississippi,  and  Nevada.  At  each 
location,  we  met  with  the  rcris  database  administrators  and  other  system 
staff,  as  well  as  officials  who  were  responsible  for  managing  and 
implementing  the  rcra  program.  We  also  reviewed  numerous  documents 
related  to  rcris  use,  operation,  and  output. 

We  reviewed  detailed  information  on  the  rcra  program,  epa’s  previous 
attempts  to  collect  hazardous  waste  data  through  automation,  and  the 
development  and  operation  history  of  rcris;  and  examined  prior  gao  and 
epa  Inspector  General  reports  that  discussed  weaknesses  in  epa’s 
information  systems  environment.  We  interviewed  epa  headquarters 
officials  from  the  Office  of  Solid  Waste  and  the  Office  of  Enforcement  and 
Compliance  Assurance.  These  officials  included  the  Director  of  the  Office 
of  Solid  Waste,  the  rcris  system  manager,  and  staff  involved  in  rcra  and 
rcris  activities.  We  also  interviewed  representatives  from  the  National 
Governor’s  Association  and  reviewed  documents  pertaining  to  its 
involvement  in  assisting  epa  to  identify  the  objectives  and  requirements  in 
rcris. 

In  addition,  we  used  relevant  segments  of  the  information  systems  audit 
methodology  published  by  the  EDP  Auditors  Foundation  to  evaluate  the 
extent  to  which  rcris’  development  and  operation  met  generally  accepted 
systems  practices.1  We  reviewed  and  analyzed  documents  on  epa’s  actions, 
including  documentation  on  user  requirements,  system  feasibility, 
functional  requirements,  change  control  procedures,  and  data 
management.  We  also  reviewed  epa  planning  documents  including 
information  on  an  Office  of  Solid  Waste  strategic  initiative. 

Our  work  was  performed  at  several  offices  at  epa  headquarters  including 
the  Office  of  Solid  Waste,  the  Office  of  Enforcement  and  Compliance 
Assurance,  and  Office  of  Inspector  General.  These  offices  were  located  in 


Computerized  Information  Systems  Audit  Manual,  EDP  Auditors  Foundation,  Inc.,  1992. 
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Washington,  D.C.,  and  Arlington,  Virginia.  We  also  worked  at  epa  regional 
offices  in  Atlanta,  Georgia;  Dallas,  Texas;  and  San  Francisco,  California; 
and  at  state  offices  responsible  for  rcra  in  Tallahassee,  Florida;  Jackson, 
Mississippi;  Little  Rock,  Arkansas;  Austin,  Texas;  Carson  City,  Nevada;  and 
Sacramento,  California. 

We  conducted  our  review  from  August  1994  through  May  1995,  in 
accordance  with  generally  accepted  government  auditing  standards.  We 
requested  comments  on  a  draft  of  this  report  from  the  Administrator  of 
epa.  The  Director  for  the  Office  of  Solid  Waste  provided  us  with  comments 
on  July  18, 1995.  We  have  incorporated  these  comments  where 
appropriate. 
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Error  Rates  for  14  Selected  Resource 
Conservation  and  Recovery  Information 
System  Data  Elements 


Actual 

error 

Range  of 
possible  error 
rates® 

Data  element 

Number 
of  errors 

rate 

percent 

From 

percent 

To 

percent 

1.  Compliance  evaluation  inspections 

16 

53 

36.68 

69.52 

2.  Comprehensive  groundwater  monitoring 
evaluations 

6 

20 

8.48 

36.49 

3.  Operation  and  maintenance  inspections 

6 

20 

8.48 

36.49 

4.  Cumulative  groundwater  violations 

11 

37 

21.68 

53.71 

5.  Informal  enforcement  actions 

13 

43 

27.48 

60.18 

6.  Formal  enforcement  actions 

7 

23 

10.9 

40.08 

7.  Assessed  penalties 

7 

23 

10.9 

40.08 

8.  Outstanding  groundwater  violations 

9 

30 

16.1 

47.00 

9.  Facility  operational  status 

5 

17 

6.21 

32.85 

10.  Environmental  ranking 

1 

3 

0.12 

16.57 

11.  Stabilization  evaluation 

2 

6 

0.91 

21.02 

12.  RCRA  facility  assessment  completed 

4 

13 

4.13 

29.07 

13.  RCRA  facility  investigation  needed 

4 

13 

4.13 

29.07 

14.  Corrective  action  status 

2 

6 

0.91 

21.02 

aLower  and  upper  bounds  of  the  95-percent  confidence  interval  for  percent  of  entries  in  error. 
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